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ABSTRACT 



This document describes the development of an instrument 
designed to assess the implementation of a professional learning community 
among school staff. It describes the growing need for such instruments as 
schools increasingly deviate from traditional approaches in education in 
their efforts to enhance student learning. An instrument that could be used 
as a screening or assessment tool to ascertain the maturity of staffs as 
learning communities could help researchers conduct studies of schools that 
were clear examples of communities of professional learners, providing 
insights on how a professional learning community is created in a school. 

Such an instrument could be used to collect baseline data to determine if the 
development of a school staff resulted in a community of professional 
learners, to explore the pacing and time required by different schools in 
different contexts, and to employ the instrument as a diagnostic tool. The 
article outlines the structure of the instrument, which was initially titled 
"Descriptors of Professional Learning Communities, " and consists of 17 
descriptors grouped into five major areas of dimensions. It details the pilot 
and field testing of the instrument and provides results of data analyses, 
descriptive analysis, reliability analyses, and validity analyses. (Contains 
10 references.) (RJM) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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For decades schools have been urged to 
engage in change and improvement activities. In 
the seventies, research on effective schools and 
on the process through which to achieve them 
was on “improvers” minds. In the eighties, the 
principal came under scrutiny for the role that a 
person in this position could play in school change 
efforts. In succession came attention to 
restructuring, school-based management, total 
quality management, and a myriad of additional 
programs and processes that were offered to 
schools to support their improvement efforts. 

The Southwest Educational Development 
Laboratory (SEDL), like many of the ten regional 
laboratories, invested staff and resources in 
studying and supporting schools in their efforts 
to improve their effectiveness so that students 
might become more successful learners. In the 
process of conducting studies that examined 
leadership strategies and contextual factors that 
might influence school change, SEDL researchers 
discovered a school whose staff worked together 
in a way that was very different from that of the 
typical school. This school became the target of 
intense exploration for three years (Hord & Boyd, 
1995). 

This school staffs vision and continuous 
conversation focused on students and student 
learning. Beginning in the early 1980s the staff 
came together regularly and frequently to reflect, 
inquire into what they were doing and how they 
were doing it, and assess whether their work was 
producing the results they wanted for students. 
They brought new ideas for programs and 
practices that would support their work with 
students, and they committed to learning new 
strategies and content to this end. 

In short, they were a staff who collectively 
searched for ways to become more effective 
teachers, who valued changing their own 
knowledge base and skills, and who sought change 



to accomplish improvement. The staff pursued 
these strategies through a decade and a half and 
a succession of four principals. 

In 1990, Peter Senge’s Fifth Discipline 
swept boardrooms in the private sector and, 
subsequently, school boards in education settings. 
Senge’s book promoted the idea of a work 
environment where employees engaged as teams, 
developing a shared vision to guide their work, 
operating collaboratively to produce a better 
product, and evaluating their output. 

Simultaneously, Susan Rosenholtz (1989) 
published ideas from her research that described 
a workplace for teachers that encouraged 
collaboration, an environment in which teachers 
shared ideas and solutions to problems, and shared 
learning about educational practice. Importantly, 
she found that as teachers learned from each 
other and improved their practice, benefits to 
students increased. 

The convergence of these literatures and 
the research that was being done in the school 
noted above led an R&D team at SEDL to design 
new work that would focus on creating and 
nurturing the kind of workplace and culture in 
schools that Senge, Rosenholtz, and Hord and 
Boyd had identified — a workplace where students 
and learning were the undeviating focus and the 
staff worked coUegially to achieve the desired 
results. 

The first phase of SEDL’s new work. 
Creating Communities of Continuous Inquiry 
and Improvement (CCCID, was initiated by a 
review of the literature to reveal what was already 
known about school staffs that worked in this 
way. The review was quite brief, since the 
knowledge base that focused on “professional 
learning communities” (PLC) or “communities of 
inquiry” (and we have used these terms 
interchangeably in our work) was in its infancy. 
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As Linda Darling-Hammond reported (1996), 
schools that operated in this way are few and far 
between. 

The literature described these 
organizational arrangements in schools, providing 
useful insights into what they looked like and how 
they worked, with the SEDL research school (noted 
above) viewed as being parallel to the literature 
descriptions. The literature also contained 
information about student gains that resulted 
when staff worked as a professional learning 
community. What was not in the literature was 
how the school’s administrators and teachers 
created or invented this way of working with each 
other. 

With the information derived from the 
literature, the SEDL researchers set out to find 
additional schools in the laboratory’s five-state 
region (Arkansas, Louisiana, New Mexico, 
Oklahoma, and Texas) that operated as 
communities of inquiry. The researchers felt that 
historical studies of these schools would reveal 
how they had become communities of inquirers 
and learners. 

An immediate issue was how to identify 
such schools. Colleagues across the five states 
had been asked to nominate schools for study, but 
this kind of school was far too unusual to be easily 
identified. Schools were enthusiastically 
nominated, but proved disappointingly inauthentic 
when examined. What to do? 

Evolution of an Instrument 

It would be nice to report that logic and 
rational thinking prevailed at this juncture. As a 
matter of fact, it was the angel of inspiration, 
which descended upon the SEDL researcher. 
Finding herself on a return trip from Africa with 
an eight-hour layover in the Cape Town airport, 
she passed the time by searching a late draft of the 
literature review for final editorial changes. 

Without really thinking about problem 
solving or a product, she began “messing around’’ 
structuring a rubric that would assess the presence 
or absence of the components of a professional 
learning community as identified in the review of 
the literature. The resulting instrument was 
patterned as an Innovation Configuration matrix 
or “map” (IC) (Hall & Hord, 1987 ; Hord, Rutherford, 
Huling-Austin, & Hall, 1987). In essence, it was 



designed to assess the existing degree of 
implementation of the components of a professional 
learning community in operation in a school staff. 
The IC orientation provided a measurement of 
what was actually happening. 

With no presumption of noteworthy 
accomplishment, the newly created instrument 
made it through customs and to SEDL 
headquarters in Austin, Texas. As serendipity or 
synchronicity would have it, however, there was 
an immediate and additional need for such an 
instrument. The professional learning community, 
or community of inquiry and improvement, was a 
concept that was of interest to others, notably a 
sister lab, the Appalachia Educational Laboratory 
(AEL), based in Charleston, West Virginia. 

One of AEL’s projects proposed to “establish 
a network and develop a process, both of which 
will harness the power of collective thinking and 
collegial learning for continuous improvement in 
schools” (Meehan, Orletsky, & Sattes, 1997, p. 5). 
It was believed that some aspects of the AEL 
project, such as “shared leadership,” might be 
measured by the instrument. A long-standing 
collegial relationship between SEDL and AEL 
staff supported the exchange of professional 
learning community information, including the 
newly crafted instrument. 

The Instrument’s Uses 

As already noted, a need had developed for 
identifying school staffs as professional learning 
communities. An instrument that could be used 
as a screening, filtering, or assessment tool to 
ascertain the maturity of staffs as a learning 
community would be welcomed. With such an 
instrument, researchers could conduct studies of 
schools that were clear examples of communities 
of professional learners, with the goal of learning 
how a professional learning community is created 
in a school. Obviously, an inquiry into a mature 
professional community of learners in a school 
would yield much information about the school 
staffs transformation. 

Additional uses of the instrument planned 
by the SEDL project would be as a means to collect 
baseline data and then to determine if the 
development of a school staff resulted in a 
community of professional learners. The 
instrument could be employed to reveal successful 
progress along a continuum of development, and 
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to explore the pacing and time required by different 
schools in different contexts, realizing that not all 
schools will reach the highest level of development 
at the same time or at the same rate. 

Another application would be to employ 
the instrument as a diagnostic tool, to be used as 
efforts are made to create communities of 
professional learners in other school settings. The 
external facilitator and/or internal persons at a 
site could use such formative data as a basis for 
decision making about the support and assistance 
needed to move the staff along in its evolution as 
a professional learning community. 

The literature is filled with exhortations 
about the power and desirability of teachers’ 
working collaboratively. The instrument could 
facilitate and support studies of how principals (or 
other campus and district leaders) work with staff 
and the effects of their efforts on teacher 
collaboration and efficacy. In addition, other 
factors, such as context variables, could be 
correlated with the development of the professional 
learning community (as indicated by the 
instrument) within a school. 

Finally, the instrument could be used 
during the continuation period after the 
establishment of a PLC. An assessment of each of 
the variables could ascertain whether it remained 
“robust” in the staffs working arrangements and 
could provide early warning signals that a part of 
the PLC was waning and therefore warranted 
increased attention. 

Structure of the Instrument 

The initial instrument was titled 
“Descriptors of Professional Learning 
Communities” and consisted of 17 descriptors 
grouped into five major areas or dimensions 
identified from the literature review (Hord, 1997). 
The five dimensions were: 

1. the collegial and facilitative participation 
of the principal, who shares leadership (and 
power and authority) and decision making 
with the staff (with two descriptors); 

2. a shared vision that is developed from the 
staffs unswerving commitment to students’ 
learning and that is consistently articulated 
and referenced for the staffs work (with three 
descriptors); 



3. learning that is done collectively to create 
solutions that address students’ needs (with 
five descriptors); 

4. the visitation and review of each teacher’s 
classroom practices by peers as a feedback 
and assistance activity to support individual 
and community improvement (with two 
descriptors); and 

5. physical conditions and human capacities 
that support such an operation (with five 
descriptors). 

The 17 descriptors were organized to 
illuminate the dimensions and were distributed 
unevenly (as noted above) across the five 
dimensions. The descriptors were designed as a 
series of three statements structured along a 
continuum that would reflect most desirable or 
more mature practice of the descriptor to least 
desirable or less mature. For example, under the 
first dimension noted above, “collegial and 
facilitative participation of the principal, who 
shares leadership . . . through inviting shared 
decision making from the staff,” one of the 
descriptors is presented as a series of three 
statements along a continuum: 

Administrator(s) involves the entire staff. 
Administrator(s) involves a small committee, 
council, or team of staff. 

Administrator(s) does not involve any staff. 

'These statements were to differentiate the 
high, middle, and low parameters of the descriptor 
along a five-point scale. The format and layout of 
the instrument required the respondent to read 
all three indicators for each of the 17 descriptors 
and then mark the response scale. 'This format 
required more mental processing than usual for a 
selected-response, Likert-type instrument, but 
contributed much to the use of the instrument as 
a screening or filtering device (see Figure 1, page 
4). 
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FORMAT OF THE INSTRUMENT 
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A Note to the Readers 

As noted earlier in this paper, the 
instrument was shared with staff at AEL. Study, 
conversation, and other interaction between the 
SEDL qualitative researcher and the AEL 
quantitative evaluator resulted in a working 
agreement: the SEDL instrument would be made 
available and used extensively in diverse school 
settings, and AEL would conduct the statistical 
processing to test the instrument and assess its 
psychometric properties. 

At this point it is important to provide a 
note to our Issues readers. The purpose of this 
paper is awareness — to let our colleagues know 
that the instrument exists. Because of the broad 
clientele of our audience, an effort has been made: 

1. to keep the language and terminology 
reasonably understandable to those who 
are not experts in instrument design and 
testing, but also 

2. to present information about the 
instrument in such a way that it is credible 
to those who are instrument aficionados 
and quantitatively oriented. The challenge 
has been to serve these two purposes. 

Those readers who are not interested in 
the psychometric testing of the instrument, may 
skip to page 7 for Conclusions about the statistical 
tests. Those who are more keen about 
psychometrics, should read on, understanding 
that it is not the intention of this Issues paper to 
present the full range of procedures and results 
conducted in the field test on the instrument. 
Those interested in the full report may see Meehan, 
Orletsky, and Sattes (1997). Those concerned 
about the robustness of the field test, its procedures, 
and reporting, should review FY97 Report- 
External evaluation of the Appalachia Educational 
Laboratory (1998). 

In this report, external evaluators scored the AEL field test 
study with two ratings of “outstanding” (on Utility and 
Accuracy) and two ratings of “satisfactory” (on Feasibility 
and Propriety), based on national evaluation standards. 

Pilot Test of the Instrument 

A small pilot test (n = 28 students, parents, 
and educators participating in a summer AEL 



project conference) was conducted in 1996, with 
encouraging results. This sample represented 
individuals much like those in any school 
community, and the results indicated that the 
instrument can be applied to a spectrum of people 
who have varying experiences and involvement in 
a learning community of professionals. 

It is important to assess the reliability or 
consistency of an instrument. There are two types 
of reliability: internal consistency (e.g., Cronbach’s 
Alpha) and stability (test-retest). For the pilot 
test, Cronbach’s Alpha reliability for the total of 
the 17 items was + .92. There is general agreement 
that + .75 or above indicates appropriate 
instrument internal consistency. 

The test-retest measures stability over time 
and the reliabilities for the 15 participants who 
could be matched with individual ID numbers was 
+ .94. The correlation of the total score of this 
instrument with the total score of a school climate 
instrument titled the “School Climate 
Questionnaire” (Manning, Curtis, & McMillen, 
1996) and deemed to assess similar characteristics 
was + .82. 

This pilot test of the instrument in the 
AEL region with a small heterogeneous group 
suggested that the instrument possessed 
pyschometric properties sufficient to continue its 
use, but a field test with a larger sample of schools 
was required. 

Field Test of the Instrument 

The field test was designed with three 
objectives for study: 

1. to assess the reliability of the professional 
learning community instrument, 

2. to assess the validity of the professional 
learning community instrument, and 

3. to draw conclusions about its use in 
educational improvement efforts at the school 
level. 

Sample 

The sample for the study included all the 
teachers in 21 schools in AEL’s four-state region 
who completed and returned the instrument. The 
schools volunteered to participate in the study 
with no external rewards or motivation offers. 
These schools were nominated to participate 
usually through the building principal or other 
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contact persons familiar with the school and its 
staff. A total of 690 teachers completed and 
returned the instrument. 

The field test schools were in Kentucky, 
Tennessee, Virginia, and West Virginia. The 
schools represented the elementary level (n = 6), 
middle/junior high (n = 6), and high school (n = 9). 
The schools’ student enrollment ranged from a 
low of 205 to a high of 1,200. The percent of 
students on free and reduced lunches in the 21 
districts ranged from a low of 12% to a high of 39%, 
with a mean of 22.5%. 

A subsample of teachers in four large high 
schools in Tennessee were involved in the AEL 
project noted above (the 4 high schools were 
included in the 21 schools in the total sample). 
They volunteered to participate also in the 
concurrent validity and stability (test-retest) 
reliability analyses by (1) completing a school 
climate instrument at the same time and (2) 
including an individual identification number on 
their instruments, for purposes of the retest. The 
number of teachers in the high schools was 53, 57, 
61, and 60. 

The four high schools are in the same 
district. The district’s student population is 99% 
Caucasian, with 13% on free or reduced lunches. 
It is reported that 64% of these high school students 
are college-bound, a figure based on the percentage 
of the 1996 graduating class that enrolled in two- 
or four-year colleges. 

Finally, in addition to being used in the 21 
AEL region schools in the field test, the instrument 
was administered to the school staff known from 
previous research and described to be operating 
as a professional learning community (the school 
referred to in the first section of this paper). This 
school, a “known group” for the construct validity 
analysis, is an urban school of 23 teachers and 
about 400 students in the New Orleans school 
district. The instrument was administered to this 
school’s staff as part of the field test. Nineteen 
copies of the instrument were sent to AEL for the 
“known group” analysis, but not every teacher 
completed every item. 

Data Analyses 

Analyses of the instrument began with a 
file of the 690 teachers in the 2 1 schools, with files 
of data from the 4 high schools, and with the file of 



the “known group.” The analyses of these files are 
presented below in paragraphs describing the 
descriptive statistics, the reliability analyses, and 
the validity analyses. All of the analyses were 
completed at AEL, using the SPSS statistical 
analysis software package. 

Descriptive analysis. Descriptive 
analysis of the 690-case file was the first step 
completed. All of the descriptive statistics for the 
1 7 individual instrument descriptor items and the 
total score were computed. Next, those same 
descriptive statistics were computed by school 
level — elementary, middle/junior high, and high 
school. Then, as one measure of the usability of 
the instrument, these same descriptive statistics 
were computed for the 21 different schools in the 
field test. 

Based on the descriptive statistics from 
the instrument with 21 schools in the AEL I’egion 
and using mean scores, the instrument does 
differentiate among all the schools. When the 
schools are subgrouped into three levels — 
elementary, middle/junior high, and high school 
— the instrument also differentiates the school 
faculties in terms of their development as 
professional learning communities. 

Reliability analyses. Reliability analyses 
consisted of two types — internal consistency and 
stability (or test-retest). First, the internal 
consistency reliability coefficient, using Cronbach’s 
Alpha formula, was computed for the total 
instrument. The Alpha reliability coefficient was 
computed on the main file of 690 cases, although 
not all teachers completed all items: it was .94. 
Next, the instrument’s Alpha reliabilities were 
computed for the 21 individual schools in the field 
test. These analyses were conducted to assess the 
reliabilities at the level of intended use — the 
individual school. These Alphas ranged from .62 
to .95, with one in the .60s, none in the .70s, 7 in 
the .80s, and 13 in the .90s. 

The instrument yielded satisfactory 
internal consistency (coefficient Alpha) reliabilities 
for the total instrument in the field test. These 
satisfactory Cronbach’s Alpha reliabilities were 
evident at both the full group and the individual 
school level. There was no pattern in the Cronbach 
Alpha reliabilities by the three levels — 
elementary, middle/junior high, and high school. 
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Second, the stability (test- retest) reliability 
coefficient was computed with the subsample of 
four high school faculties in Tennessee. Because 
of a problem in matching unique identification 
numbers, the number of cases that were usable 
was low (n = 23). Even though the coefficient of 
stability reliability value was computed on a 
smaller than ideal subsample, the resulting value 
for the total instrument score (.6147) was 
marginally satisfactory, with the potential to 
increase, or decrease, if the sample size were to 
increase. 

Validity analyses. Validity analyses 
consisted of three types — content, concurrent, 
and construct (two methods). 

First, content validity (checking that the 
content is appropriate) was assessed at three 
stages: duringthe development, early review, and 
modest reformatting of the instrument. In the 
first stage, the content of the five dimensions was 
established by the author from her review of the 
educational and business/corporate literature 
(Hord, 1 997) , plus her field research with southwest 
U.S. schools that functioned as professional 
learning communities. The second stage of the 
content validity assessment was conducted by 
three AEL staff as they independently reviewed 
the five dimensions and 17 descriptors. They 
modestly reformatted the instrument after 
reaching consensus on wording to gain additional 
clarity and consistency. AEL sent the reformatted 
instrument to the author, and the third stage of 
content review was completed when the author 
assessed the minor word changes and confirmed 
that the reformatting was consistent with the 
original intentions for the instrument. Based on 
the three stages of the review of the items in the 
instrument, the instrument was judged to possess 
sufficient content validity for its original intention 
of measuring the concept of a community of learners 
within the professional staff of K-12 schools. 

Second, concurrent vahdity (comparing the 
instrument with another purporting to measure 
the same concept) was assessed by administering 
a school climate instrument. With respect to the 
concurrent validity, the instrument possesses 
satisfactory correlation with the school climate 
instrument used in the field test with a subsample 
(n = 114) of four high school faculties (the 
correlation between the 17-item field test 
instrument and the 10-item school climate 



instrument was .7489, significant at the .001 
level). 

Third, construct validity asks the question. 
Does the instrument measure the psychological 
construct called “professional learning 
community”? The “known group,” noted earlier in 
this paper, was the first method used for construct 
validity analysis. The scores of the teachers in the 
school that was known from previous research to 
be functioning as a professional learning 
community were compared to the scores of the 690 
teachers from the 21 schools in the field-test 
database. 

The 21 AEL schools were volunteer schools, 
and no assumptions were made as to whether or 
not they were schools of professional learning 
communities; no data were available to support or 
to refute that. The purpose of this construct 
validity check was to assess the difference of the 
scores from the known-group teachers with the 
scores from all other teachers in the main database 
with the t-test. The higher scores from the teachers 
in the school that was known to be a learning 
community of professionals differ significantly 
(.0001) from those of the teachers in the field test. 
Using the known-group methodology, the 
instrument appears to represent the construct of 
a mature professional learning community. 

Last, factor analysis, the second method of 
construct validity analysis, included unconstrained 
principal components analysis followed by both 
varimax and oblique rotations of the data. The 
final factor analysis solution was an iterative 
process of comparing the before-rotation data with 
the after-rotation data, then going back to the 
descriptive statistics on the scores, and including 
their distributions. Based on factor analysis 
results, it appears that the 17-item instrument 
represents a unitary construct of a professional 
learning community within schools. 

Conclusions 

As noted, the usability, reliability, and 
validity tests have been completed, and the 
instrument appears to meet the expected criteria. 
To summarize, the tests for reliability and validity 
have been satisfactorily met. After testing of the 
instrument it was concluded that, overall, it 
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appears that the 1 7 -item instrument is very useful 
as a screening, filtering, or measuring device to 
assess the maturity of a school’s professional staff 
as a learning community. 

The instrument testing has been expanded 
to a national field test — that is, it is being utilized 
by individuals across the nation. From these 
individuals, school staff responses are being 
collected and analyses conducted to enhance and 
strengthen the database. After the current 
national field testing, the instrument is expected 
to be available for dissemination and widespread 
use by educators and others as a diagnostic or 
assessment tool that can support the nurture and 
development of professional learning communities, 
and subsequently, school staffs’ collegial work for 
continuous school improvement. 
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